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Abstract. In this paper, we consider systems of semilinear elliptic equations 

-A m nu=\v\ p - 1 v, 

(0.1) 

-A H ivi; = \u\ q u, 

in the whole of Hyperbolic space M. N . We establish decay estimates and symmetry properties 
of positive solutions. Unlike the corresponding problem in Euclidean space K , we prove 
that there exists a nonnegative nontrivial solution of problem (jO.ip . 
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1. Introduction and main result 

In this article, we will study decay, symmetry and existence of solutions of the following 
semilinear elliptic systems 

-A m nu = \v\ p ~ l v, , , 

-A H ^ = \u\ q -\ { ' 

on Hyperbolic space H , where A H iv denotes the Laplace-Betrami operator on M. N , N > 3,p 
and q satisfy a suitable condition. 

When posed in the Euclidean space M. N , problem (II. ip has two features. First, it is the 
Emden-Fowler equation 

- Au = \u\ p ~ l u in R N . (1.2) 



Such a problem has been extensively studied, see for instance|8j|9j [12j[13j[l4] and references 
therein. Attention was focused on the existence and Liouville-type theorem for solutions of 
problem (ll.2p . There is a host of later important contributions to the subject, among them 
we must mention the famous paper by [14] where the Liouville-type theorem of problem (II. 2p 
was obtained. They proved that the only non-negative solution of ( II .2p is u = when 

iV + 2 

Second, it is the Hamiltonian type system 

-Au = M P ~V in R N , n „N 

-Av = M ? -y in R N . 1 j 

Using a blow up technique, JieQing [19] and M.A.Souto [25jhas been established the priori 
estimates for solutions of problem ( II. 3p . In [10], Figueidedo and Felmer proved that if 
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p > 0, q > are such that 

N + 2 , , 7V + 2 

P,q < -jy — r' u ^ n °t °° are e 1 U( d to — — -, 

then the only non- negative solution of (11.31) is the trivial one u = 0,v = 0. For more 
general nonlinear elliptic equations in the Euclidean space ~R N , we refer to [6j[7j[21j[22j[23j 
and reference therein. 

It is also of interest to study problem (11.21) and (11.31) with respect to different ambient 
geometries in particular to see how curvature properties affect the existence and nature 
of solutions. A recent paper by Mancini and Sandeep[20j have studied the existence / 
nonexistence and uniqueness of positive solution of the following elliptic equation 

-A m nu = \u\ p ~ 1 u + Xu (1.4) 

on Hyperbolic space M N . They proved that if A = and 1 < p < jp^, then problem (11.41) 
has a positive solution. This result is contrasted with the result in Euclidean space due to 
P3]. Afterward, Bhakta and Sandeep [3] have investigated the priori estimates, existence 
of radial sign changing solutions of problem ( ll.4p . In jl], classification of radial solutions is 
done by Bonforte etc for problem ( 11. 4p . 

Our aim in this paper is to study the decay, symmetry and existence of solution of problem 
(11. ip . Our result should be contrasted with a result due to Figueidedo and FelmerfTO]. The 
difficulties in treating system ( II .ip originate in at least three facts. First, there is a lack of 
compactness due to the fact that we are working in which is a noncompact manifold. 
Second, due to the type of growth of the nonlinear term, we can not work with the usual 
space i^ 1 (H iV ) and then we need inhomogeneous Sobolev space. Third, although, we have a 
variational problem, the functional associated to it is strongly indefinite. 

Now we are ready to state our main result. In section 2, we discuss the symmetry property 
of positive solution of problem ( II. ip . 

Theorem 1.1. Forp,q satisfying 

N + 2 . . 

P,g<^3^ (1.5) 

Then, all positive solutions (u,v) G H 1 (M N ) x H 1 {M N ) of problem U.l\) are Hyperbolic 
symmetry, i.e. there is xq G H , such that (u,v) is constant on hyperbolic spheres centered 
at Xq. 

In section 3, we prove a result on the decay of solutions of problem ( II. ip as |x| — > oo and 
in fact, exponential decay. The results of Section 3 is the following. 

Theorem 1.2. Forp,q fulfilling M.5\) . let (u,v) G H 1 (M N ) x H 1 (M N ) be a positive radial 
solution of M.l\) . then u'(r) < 0, v'{r) < for r > and 



Moreover, 



lim u(r) = lim v(r) = lim u'(r) = lim v'(r) = 0. 

r—¥oo r— >oo r— >oo r— >oo 



log?/ 2 \ogV 2 \ogu' 2 log?/ 2 

lim = lim = lim = lim = — 2(iV — 1). 

r— >oo f r— >oo f r— >oo f r— s>oo f 



In section 4, we prove the existence of radial symmetric solutions of problem ( 11. ip . Namely 
the following theorem. 
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Theorem 1.3. Suppose that (p,q) satisfying 

1 1 N-2 

p+l + q+l > N ' 
then problem U.l\) has at least one radial solution (u,v). 

Finally in section 5, we prove the existence of a ground state solution for the system (II. ip . 

Theorem 1.4. Suppose that (p,q) satisfying ( fi.51) . then problem ( fi.il) has a nonnegative 
nontrivial ground state solutions. 

2. Proof of theorem 11.11 

The main purpose in this section is to prove Hyperbolic symmetry properties of solutions 
of (11. ip . The way of proving this symmetry is by move planes, as originally introduced by 
Alex androff[18j, later used by Serrin[24"]. and extensively used in recent times after the world 
of Gidas-Ni-Nirenberg|12j . The case on noncompact manifold was studied in [T][2]- 

Proof of Theorem ll.lt 

Let A t be a one-parameter group of isometries of which is C7 1 (M x H , H ) and I be 
a reflection (i.e. I is an isometry and J 2 =Identity) satisfying the invariance condition 

A t I A t = I, W G R. 

We translate the reflection / using A t to define a one-parameter family of reflections 

It = A t IA_ t . 

Let Ut be the Hyper-surface of which is fixed by I t . We also assume that (J U t is 

h<t<t 2 

open for all h, t 2 e R, and \jU t =M N . Forte R, define 

Qt= |J U s , Q l = |J U a , 

— oo<s<t t<s<oo 

then we have h(Qt) C Q*,and It{Q l ) C for all tel. For i 6 K and x G we define 
x t = It(x), and 

u t (x) = u(I t {x)), v t {x) = v(I t (x)). 

Define 

A = {t G R : Vr > t, u > u T , and v > v T , in Q T }, 

the first step of proof is to show that the set A is non-empty the second step is to prove 
that A is bounded from below. Finally we will show that if A = inf A, then u = u\, v = 
in Q K . 

Step 1. A is non-empty. 

Since (u,v) G i? 1 (EI Ar ) x i/ 1 (H Af ), we can take (u\ — u) + or (v\ — v) + as text function. 
Then we obtain 

= Sq^vxI^vx ~ \v\ p - l v)(u x - u)+ dV M N 

< (I Qx \vx\ p+1 dv m ) p ^(j Qx |K - vyr 1 dv mN )^(j Qx \( Ux - ^r 1 dV uN )^ 

< c(J Qx dV uN ) P ^(J Qx \V m n(v x - v)+\ 2 dV mN ) l 2(J Qx \V b n(u x - u) + \ 2 dV BN )h, 
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and 

J 0l |Vffv(vA - v) + \ 2 dV M N 



Qx 

< c(J Qx Kl 9+1 dV aN )^(J Qx \V m n(u x - u) + \ 2 dV mN )m Qx \V m m(v x - v)+\ 2 dV mN )l. 
It implies that 

(I Qx |V h ^(ma - u) + \ 2 dV mN )h 
< <f Qx \vx\ p+1 dV m )^(f Qx \V m 4vx-v) + \ 2 dV mN )l 



and 



Thus 



(J . |V H iv(vA - v) + \ 2 dV W N)2 



'Qx 

< c(j Qx Kr 1 dV m ) 9 ^(J Qx \V m (u x - u)+\ 2 dV m )l 



[l-c(f \u x \ q+1 dV m ) 9 ^(f \v x \ p+1 dV m )^]( [ \V mN (u x -u) + \ 2 dV m )^ <0. 
JQx JQx JQx 

and 

[l-c(/ \u x \" +1 dV mN )^([ \v x \ p+1 dV BN ) P ^]( [ \V m n(v x -v) + \ 2 dV mN )^ <0. 
-/<2a -/<3a -/<2a 

Since we can choose Ai G R, such that 

c([ \u x \« +1 dV m )^([ \v x \ p+1 dV m )^ < 1, 

-'Qa -'Qa 
Then for any A > Ai, we have 

|V H iv(wA - u) + \ 2 dV M N < 0, 



Qx 



and 



/ \V m n(v x -v) + \ 2 dV mN < 0. 



'Qx 

Therefore (u x — u) + = 0, and (i>a — f ) + = in Q x , and (Ai, +oo) C A. 
Step 2. A is bounded from below. 
By (u,v) G if 1 (IF) x ff 1 ^), we have 

lim supw = lim supf = 0, 
and we may choose A2 such that 

sup u sup v 

sup u < , sup V < . 

Qx 2 2 Qa 2 2 

This implies that all A G (—00, A 2 ) do not belong to A. Therefore A is bounded from below, 
and we let A = inf A. 

Step 3. u = u A , v = t>A, in Qa- 

In fact, it is clear that by continuity of the foliation and of (u, v), we have 

u > u A , v > v A , in Qa- 
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Now observing that if u = u&, it follows from 

-A M n(u-U K ) = \v\ P ^V - |fA| P ~V, { ,-y j , 

-A H iv(w - w A ) = - |ma| 9_1 MA) 



we can get v = v\. So if we assume, by contradiction that the step 3 is not true, we conclude 
that 

u > ma? v >v^ for x G Q\, 
By the strong maximum principle and connectedness of Qx implying that 



(2.2) 



v > ua, for x G <5 A , 
and 

X(it)(x) > 0, for x G [/a, 



(2.3) 



(2.4) 



X(v)(x) > 0, for x G L?a, 

where X is the killing vector field associated to the transformation group A t , we shall see 
that this is impossible. 

Choose x\ G [/a, and R > 0. By the continuity of the foliation, there would exist e > 0, 
such that for < e < £q, I&_ e (B(xi, Rq)) C B(x\,2Rq). Moreover, by the definition of A, 
we could construct an increasing sequence A n > A, such that A n > A — Eq, and 3y n G Q\„, 
such that 

u{y n ) < u{I Xn {y n )) = u Xn {y n ), 

or 

v(y n ) < v{I Xn {Vn)) = v\ n {y n ). 

We claim that y n G B(x\,2Rq). If it is not true taking (u\ n — u) + or (v\ n — v) + as test 
function(as in the first step). We would have that u\ n < u, or v\ n < v, in Q\ n . This proves 
our claim. 

Modulo a subsequence. There would exist y G Qaj sucn that y n — > y. By continuity we 
have 

u(y) = lim u(y n ) < lim u Xn (y n ) = u- K {y) , 

n— >+oo n— >+oo 

or 

= lim < lim v An (?/n) = v~ K (y). 

n— >+oo n— >+oo 

It implies that t/ G J/a- On the other hand, there exist points £ n in the line segment between 
y n and I Xn (y n ), such that X(u)(£ n ) < 0, passing to the limit we should have X(u)(y) < 0. 
This is impossible X(u)(x) > 0, for all x G Hence u = u^, and v = in Qa- D 



3. Decay estimates 

The hyperbolic N-space ET^, > 2 is a complete simple connected Riemannian manifold 
having constant sectional curvature equal to -1, and for a given dimensional number, any 
two such spaces are isometric [28J. There are several models for M N , the most important 
being the half-space model, the ball model, and the hyperboloid or Lorentz model. 
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Let = {x G M N : [re j < 1} denotes the unit disc in ~R N . The space M N endowed with 
the Riemannian metric g given by = ( 1 _^| 2 ) 2 ^j is called the ball model of the Hyperbolic 
space. The hyperbolic Laplacian A m n is given by 



A 



.1 — \x 



-) 2 A + (iV-2)lJ^(x,V), 



2 ' v ' 2 
Let (u. v) be a positive symmetric solution of problem f JTTj) . and u = u(\£\), v = v(\£\), |£| < 
1, then 



1 -|ei\ 2A .. . , AT ox 1 - 



(3.1) 



! Av + (N-2) 



(£,Vv> + u* = 0, 



• 2 7 v 7 2 

Setting |£| = tanh|, iz(t) = it(tanh|), v(t) = u(tanh|), k(t) = (sinht)^ -1 , it is easy to 
see that 



\u 



W N 



W N 



W N 



1+ 1 dV m = wjv-i / k(t)\u\ q+1 dt 
Jo 

poo 

\v\ p+1 dV m = w N ^ I k(t)\v\ p+1 dt 
Veivu| 2 dV m N = tojv_ 
V h jvi>| 2 dV M N = wn- 



k(t)\u'\ 2 dt, 
k{t)\v'\ 2 dt, 



where u>at_i denotes the surface area of S N 1 . 
In addition, (13. ip rewrites 



u 



N -1 
tanht 



u' + v p = o, 



as well as 



N — 1 

i;" + ^-V + = 0, 
tanht 

{ u '(0) = v'(0) = 0. 

' {k{t)u')' + k{t)v p = 0, 

(k(t)v'y + k{t)u q = 0, 

u'(0) = i/(0) = 
and if («,«) G fl' 1 (H JV ) x iJ 1 (H Ar ) solves ([53[) . then 

k(t)u'v'dt= / k(t)u q+l dt= / k{t)v p+1 dt 



Now, let us notice that (it, u ) solves (13.21) . and 



^(u,«)(*) = H — r + 



p + 1 g + 1 



(3.2) 



(3.3) 
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then 

dtJ(u,v)(t) 

= u"v' + u'v" + v'v p + u'u q 
= (u" + v p )v' + u'(v" + u q ) 

_ _ N-l y /2 N-l u >2 



tanh t tanh t 

< 0. Vt > 0. 

Lemma 3.1. Let (u,v) G H 1 ^) x H 1 ^) be a positive solution of / TOP , then u'(t) < 
0, v'(t) < 0, for every t > 0, and 

lim u(t) = lim v (t) = lim u'(t) = lim v'(t) = 0. 

t— >+oo i— >+oo t— >+oo t— >+oo 

Proof. By equation (Q, we have (k(t)u'(t))' < 0, and (k(t)v'(t))' < 0, and from tt'(0) = 
1/(0) = 0, then u'(£) < 0, v'{t) < 0, Vt > 0. 
In particular, by u(i) > 0, i>(t) > 0, it exist 

ii(oo) = lim u(t), v(oo) = lim v(t). 

t—t+oo t— >+oo 

Since </(„„') is decreasing, then lim u'(t)v'(t) exists. 

t—>+oo 

Since (u,v) G ff 1 ^) x /^(HF), then 

liminf k{t)[u' 2 {t) +u 2 {t)] = 0, 

t— >+oo 



and 



Thus, we have that 



liminf k{t)[v' 2 {t) +v 2 {t)} = 0. 

t— >-+oo 



lim u(t) = lim t>(£) = lim u'(t) = lim = 0. 

t— >+oo t^f+OO t— >+oo t— >+oo 



□ 



Lemma 3.2. Let (u,v) be a positive solution of problem M.l\) , and (u,v) G i/ 1 (H 7V ) x 
H l {M N ), then 

hgu 2 \ogv 2 hgu' 2 \ogv' 2 

lim = lim = lim = lim = —2{J\ — 1). 

t— >+oo t t— >+oo t t^+oo t t— >+oo t 



Proof. By (13. ip . we obtain 



lim u(t) = lim v(t) = 0. 

t— >+oo t—>+oo 



Then it exists t £ > 0, such that 

cotht < 1 + e, v p (t) < ev(t), u q {t) < eu(t), Vt > t £ . 
Since u'{t) < 0, v'(t) < 0, we have for t >t £ , 

u" + {N -!){! + e)u < u" + (N - 1) cotW + v p = 0, 

(3.4) 

u" + (N - l)u' + ev> u" + (N - 1) cothtu' + v p = 0. 
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and 

v" + (N - 1)(1 + e)v' < v" + (N-1) cothto' + u q = 0, 

(3.5) 

u" + (JV - l)t/ + eu < v" + (N - 1) catW + u q = 0. 
Then, we get 

{u + v)" + (N — 1)(1 + e)(u + v)' < (u + v)" + (N - 1) cotht(w + «)' + ^ + ^ = 0, 

(u + v)' + (N — l)(u + v)' + e(u + v) > (u + v)" + (N - 1) cotht(w + «)' -)- it^ + — 0. 
It implies that 

(u + v)" + {N -!)(! + e)(u + v)' <{u + vf + {N-l){u + v)' + e(u + v). (3.6) 



Let /x"(e) = -(TV - 1)(1 + e), //+(e) = 0, and = ^ 1} ^ (jV 1} — , i/+(e) = 

(jv k e characteristic roots of the differential Polinamials in the L.h.S and 

R.h.S of (13. 6p respectively. We choose e < , then u ± (e) is real and distinct. Similar 

as [20]. we can get 



u{t) + v{t) > {[u{t £ ) + v{t E )e^-^])e^^\ V* > t £ , 



(3.7) 



(3.8) 



u(t) + v{t) < ([u(t £ ) + v{t e )e- v ~^]y~^ t , Vt > t e , . 
" u'(t) + i/(r) > A/-(e)(u(r) + u(r)), Vr > t £ , 

u'(r) + t/(t) < i/-(e)(«(r) + u(r)), Vr > t e , . 
We see from (13.71) that 

-/N^ r • f log(w + w) 2 log(u + v) 2 w^n 
2/x (e) < hmmf < hmsup < 2u (e), ve > 0, 

t^+co t t^+oo t 

and hence 

lim M!i±^ = _ 2 (JV-l), (3.9) 
From ( 13 . 71) and ( 13. 81) . we also get 

hm l ° g{u ' + V ' )2 =-2(N-l). (3.10) 
Since u'{t) < 0, < 0, we conclude from (13 . 1 Of) that 

fin !^ = -2(iV-l), 

and 

loe v' 2 

l im ^L = _ 2 (jV-l . 

t— >+oo t 

From u(t) > 0, v(t) > 0, and (13. 9ft . we also have that 

l im i^!= l im ^! = - 2 (iV-l). 

t— ¥+oo t t— >+oo t 

□ 

Proof of Theorem 11.21 By Lemma 13.11 and Lemma 13.21 Theorem 11.11 is proved. □ 
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4. Existence of radial solutions of (11.11) 

Now, we denote the functional 

I(z) = I(u,v)= V m nu-V m nv dV M N — / (v + ) p+1 dV M N — / (u + ) q+1 dV M N, 

Jm N V + 1 Ju N 1 + 1 Ju N 

which is the functional of problem (II. ip . where z = (u, v ), u + = max{«, 0}, v + = max{u, 0}. 

Observe that the quadratic part of / is well-defined, if u, v G if 1 (H Ar ). But, if we take 
such u and v, the nonlinear part is well defined if p and q are both less or equal to ^f, for 
N > 3. However, we would like to consider pairs (p, q) that do not satisfy this restriction. 
The basic requirement would be that (p, q) is below the critical hyperbola, one of them could 
be large that So, we need inhomogeneous Sobolev spaces on hyperbolic space. 

Now, we will introduce some aspects of the harmonic analysis and the geometry of hyper- 
bolic space, see [5] [15j[16j[I7][27] and reference therein. 

We consider the Minkowski space M. N+1 with the standard Minkowski metric 

{dgf = -(dx ) 2 + (d Xl ) 2 + ■■■ + (dx N ) 2 , 

and defined the bilinear form on M iV+1 x M. N+1 

[x, y] = x y - xiyi x N y N . 

The Hyperbolic space M N is defined subset of R N+1 by 

U N = {x e R N+1 | [x,x] = 1, and x > 0} 

which is the hyperboloid model. 

In geodesic polar coordinates, the Riemnanian metric is given by 

(dg) 2 = (dr) 2 + (sinhr) 2 (c^i) 2 , 

and the Riemnanian volume by 

dV^N = (sinhr^^drdVsiv-i. 

The Fourier transform (as defined by Helgason [15] ) takes suitable functions defined on 
U N to functions defined on R x S* -1 . For w e S N ~ 1 , and A G C, let b(w) = (1, w) G R N+1 , 
and 

h x , w : — > C, h X:W (x) = [x, b(w)f~ d , 
where d = ^y^. It is known that 



A H iv/i AiM , = -(A + d 2 )h x , w . (4.1) 

"IOC 

' r\ I .c/„\r_- L/_.AliA— d _7t 

This transformation admits a Fourier inversion formula, if / G L7^°(HI Ar ), then 
f(x) = F-\f) = / / f(\,w)[x,b(w)]- a - d \C(\)\- 2 d\dw, 

JO J§N-1 

where, for a suitable constant C, 

r(zA) 



The Fourier transform of / G C7o°(H ) is defined by the formula 

/(A, w) = F(f) = I f(x) [x, b(w)f x - d dV m 
Jb n 



C(A) = C 



T(p + i\) 



10 HAIYANG HE* 

is the Harish-chardrac-function on H d , and the invariant measure of S N_1 is normalized to 
1. 

Now we define the inhomogeneous Sobolev space on M N . There are two possible defini- 
tions: using the Riemannian structure or using the Fourier transform. There two definitions 
agree [26]. For p G (1, oo) and s e R, we defined the Sobolev space H S ' P {M N ) as the closure 
of C^°(M N ) under the norm 

\\f\\H^P(M N ) = IK-Ah*) 5 /||lp(h^) = \\F' 1 (X 2 + d 2 )2F(M)|| L p (H JV). 

For seK, let H S (W N ) = H S < 2 (W N ), in particular, W lj? (M N ) = H 1,P (M N ), and the Sobolev 
embedding theorem 

N N 

H s ' p L q , if 1 < p < q < oo, and s = 

p q 

holds. Moreover, for s > t, H s C H f , see [2?]. Let 

H°(U N ) = {«£ H'(E N ) : u ?s radial}, 
we have the following imbedding theorem. The case s = 1 was proved by [3]. 

Lemma 4.1. Lei s > 0, then the restriction to H*(W ) of the Sobolev imbedding of H S (M N ) 
into L r (M N ) is continuous, if2<r< , and it is compact if 2 < r < ™ 2s . 

Proof. From [3], we have that H^(M N ) ^ L P (M N ),2 < p < ^ is compact. 
Case 1. For s > 1. 

From the definition of H S (M N ), we have 

H s r {M N ) ^ Hl(M N ). 

Hence, 

ON 

H*(H. ) L P (U N ), 2<p< j^—^ is compact. 

For -J^ < 7 < n-2s i we can deduce this Lemma using Sobolev inequalities and Holder 
inequalities. Indeed, if — r < we S e ^ 

M 7 dV m < ( / dV M N) a ( [ \u\ p dV M N) l - a , 

w N Jw N Jw N 

Where a = 2N- P (N-2s) - 

Case 2. For < s < 1. 



From [27], we have that 



and 



[L 2 (U N ),H 1 (U N )} S = H S (U N ), 



[L 2 r (U N ),H},(U N )} s = H°(W N ). 
|, it(t) = w(tanh |' 

✓•oo 

|-u| 9 dVapv = ^at_i / A;(t)|w| 9 <it, 



From section 2, setting |£| = tanh|, u(t) = w(tanh|), k(t) = (sinht)^ 1 , we have 



w N 



p /*oo 

I \V^nu\ 2 dV-QN = wn-i / k(t)\u\ 2 dt. 
Jw N Jo 
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Now we claim that: 

||(sinht) ^«(*)||hi(i,oo) < C\\u\\ m{mN) , Wu e Hl(U N ). 
To do this, let v(t) = (sinht)^2 Jl -u(t), we have 

|f | = |^i(sinht)^(cosht) M (t) + (sinht) ^u'(t)\ 
<^H + (smht)V|f|. 

It implies that 

/-|f|^<C7(/-(sinht)^V(t)^ + /-(sinht)^ 1 |f|^) 
< C\\u\\ H i( a N). 

For 

POO 

/ (smht) N ~ 1 u 2 (t)dt = w^WuWliq&n), 
Jo 

and by the interpolate theory, we get 

AT— 1 

\\(smht)~u(t)\\ H »(l,oo) < C\\u\\ H .(BNy 

Thus, we can easily deduce this Lemma by Vg G (2, j^z^), 

< C\\u\\ H s {mN) , Vu e H*(W N ), 

N — l 

||(sinht)~M(t)|| Lfl (i )0o) < C||u||h.(hjv). 

Indeed, we have 

(sinh t) N -*\u(t)\ q dt < ^-n'^ fn (smh t) ^\u(t) \Ht 

— 5 — c 

< ( e fl + e --R )(J y- 1 )(j/-l) \\ U \\Hs{Wl*)- 

It implies that if R is large enough, 

poo 

w N ^ (sinh t) N - 1 \u(t)\ q dt <e, VueH°(U N ) 
Jr 

□ 



Now, let Lj:(H ) be the space of L -functions in HI which are radially symmetric. Let 

2 t"KJ\N\ 



^(H^) be the space of L 2 -functions in M. N 
T = — A h at with the domain D(T) = H 2 (M N ) which is the space of radial symmetric 
functions that are in L 2 and have second derivatives in L 2 . For < s < 2, the space E s , 
which is the domain D{T%), is precisely the space obtained by interpolation between H 2 (M N ) 
andL 2 r (M N ), 

[H 2 (M. N ) , L 2 r (M. N )] i_ | . 

In this case, the space E s is the usual Sobolev space H*(M. N ). So denoting by A = 
(— Ajjjv)^, we have for all < s < 2, 

D(A S ) = H s r (U N ). 

Let E = H?(W N ) x H*(M N ) and bilinear form: B : E x E — > R is define by 

B[(u,v),($,V)]= [ A s uA 1 ^ + A s ^A l v, 
Jw N 
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and the corresponding quadratic form by 



Q(z) = / A'uA'v, (u,v) G E. 
Jw N 

This quadratic form with replace J mN V H ivu ■ V H ivu, so we consider the functional $ : E — > 
M, defined by 

Q{z)= f A'utfv dV mN - -L- [ (v+Y +1 dV m - [ (u+y +l dV W N, 

Jm n V + 1 Jw N Q + 1 Jr n 

For z = (u, v) G E, <& is a C 1 functional and 

{<S>'( z ), y ) E = ! A s uA l m + A s ^A l v- f K) p $- / 
For 2; = (it, u) 6 .E and y = ($, G -E. So the critical points of $ satisfy the equations 
/ A'uA'V - / (v + ) p \l> = 0, /or a// * G 

and 

/ A^A'v - / = 0, for all $ G H*(H. N ). 

Jw N Jw N 

Similar as we have E = E~ © E + and z~] = for z + G £~ G where 

E~ = {{u, -A- l A s u) : u G H*(M N )}, E + = {{u, A^A'u) : u G H*(W N )}. 

We also have Q(z) = ^B[z,z] and |||^|||; = Q(z + ) — Q(z~), where z — (u, v) G E and 
z = z + + z~, z + G £ + , z~ e E~. 
Proof of Theorem 11.31 

Thanks to H*(M N ) L q (M N ), 2 < q < is compact, then $ : E — > E satisfies the 
Palais-Smale conditions as jTTJ and using the linking theorem similarly as [TTJ, we can get a 
radial solutions of problem (11. ip . 

5. Existence of Ground state solutions 
By Theorem 11.31 , we know that the set 

{(u,v) G E = H\U N ) x H 1 ^) : (u,v) is a nontrivial solution of (10)}. 
is non-empty set. We defined 

I°° = inf {I(u, v) I (u, v) is a nontrivial solution of (11. II) }. 



In order to give the proof of Theorem ll.4[ let us define 

Definition 5.1. For r > 0, define S r = {x G M. N : \x\ 2 = 1 + r 2 } and for a G S r define 

A(a,r) = B(a,r) C)U N 

where B(a, r) is the open ball in the Euclidean space with center a and radius r > 0. More- 
over, for the choice of a and r, dB(a,r) is orthogonal to S' Ar_1 . 

Similarly as |3J, we have 

Lemma 5.1. Let n > 0, r 2 > and A(a{, n),i = 1,2 be as in the above definition, then there 
exists r G X(M N ) such that r(A(ai,ri)) = A(a 2 ,r 2 ), where T(M N ) is the isometry group of 
U N . 
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Lemma 5.2. If p, q < then I°° is attained and I°° > 0. 

Proof. By Theorem 11.31 . there exists a positive solution of (II. ip . so {z G E : I'(z) = 0,2 ^ 
0} 7^ 0, and I°° is finite. If z = (u, v) is a solution of (II. ip . then 

v h nm • Vhnw rfviiv = / (u+) q+1 + K) p+1 



and hence 

I(z) = I(u,v) 

= J m V h nm ■ V M nv dV m - J HN (w + ) p+1 e^V - ^ J HN (w+) 9+1 
= (I - ^l) Jhn(^) P+1 ^ + (|-^x) Jh«(^+) 9+1 d Vw» 

> o. 

Now, we show that I°° is obtained and positive. 

Step 1. We show that the set of non-trivial solutions is bounded from below. In fact, we 
have 

IIK>+IIhi(h^) = Jw"( v +) Pu + dv ^ N 

< (J mN M p+1 dV mN )^(f mN (u + y+ 1 dV mN )^ (5.1) 

< cII^+IIlp+i^jII^+II^ 1 ^)- 

It implies that 

Ik+llfi" 1 ^) < cII^+IIlp+^hw)- 

Now, using the two equations, we obtain 

K) p+1 dV m = [ V m nu ■ V m v dV m = [ (u + )o +1 dV m . (5.2) 
From (OJ) . (l!r2j) . it follows that 



\ u +\\li+ 1 (EI n ) < ||w+||jji(H^) < CII^+lliP+lfHJV) < C\\u + 



P(g+i) 
P+l 



Thus we get 

p(g+i) 

||M+||w+i(H^) < C|| u +lli,«+i(HJV)- 

Which implies ||« + || ig +i( H Jv) 

> c, since > 1. 

Similarly, we prove that Hf+HLP+i^) > C. 

Step 2. Suppose now that z n = (u n ,v n ) is a minimizing sequence of I°°, that is 

I(z n ) — )■ J°°, J'(z n ) = 0, 2 n ^ 0. 

Clearly, {,z n } is a (PS)joo sequence for /. Now, we want to prove that {z n } is uniformly 
bounded in E. Indeed, on one hand, we have 

d - -4t) / [K) + r +i + d - -4t) / iw+r 1 dv mN < i°°, 

2 p + 1 J w n 2 q + 1 J m n 



it implies that 



[K)+] p+1 + / [K)+] 9+1 ^ < c. 

u N Ja N 
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On the other hand, 

0={I'(z n )X) 

> II4II 2 - (LA(vn) + ] p+1 dv mN )^(j mN \vtr l dV mN )^ 

-(J m A(Un)+] q+1 dV uN )^(J mN dV uN )^ 

>H4ll 2 -(/e-[K0 + F +1 

Which implies that \\z^\\ < C, similarly \\z~\\ < C, thus ||z n || = \\z^\\ + < C. 
Hence, we may assume 

z n — ► z = m £, z n — > 2 zr* ^(i^) x Lf 0C (M JV ), 

as n — > oo for any 2 < q < jfz^- 

Step 3. By step 1, we know that, there exists 5' > such that 

liminf / [K)+] 9+1 = liminf / [K)+] p+1 dV M N > 5' > 0. 

p+1 9+1 

Let us fix 5 > 0, such that < 5i < 5' < S^, 1 , < 5 2 < 5' < , where S NjP satisfies 
that 

Sn, p ( I \u\ p+1 dV m )th < [ \V m nu\ 2 dV m , \/u e H\U N ), Kp< AT > 3. 

Jm N Jm N is — z 

Let us define the concentration functiomQn : (0, oo) — > R as follows. 



Q n (r) = sup / [K) + ] 9+1 dV m . 

x&Sr JAtx-r) 



I A(x,r) 

Now, lim Q n {r) = 0, and lim Q n (r) > 5 2 as for large r. A(x,r) approximates the 

r — >0 r — >oo 

intersection of M N with a half space {y G R^ : (y,x) > 0}. Therefore, we can choose a 
sequence R n > and x n G Sr u such that 

snp / l(u n ) + ] q+1 dV mN = [ [K)+]" +1 dV mN = S 2 . 

xeS Rn JA(x,R n ) JA(Xn,Rn) 

For Xq G S^, and using Lemma , choosing T n G I(M. N ) such that 

A(x n , R n ) = T n (A(x , VS)). 

Now define z' n = (u' n , v' n ) = (u n o T n (x), v n o T n (x)). Since T n is an isometry one can easily 
see that {(u' n , v' n )} is a (PS) C sequence of I at the same level I°° as (u n ,v n ), and 



/ 

Jw 



We also have 



dV mN = [ dV mN . 
Jw N 



L ( , 0) v3)[K) + ] p+1 ^ = L^a) l[«)+F +1 ^ < sup / ■ ^ jKO+r 1 = ^ 

(5.3) 
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and \\z n \\ = \\z' n \\. Therefore, up to a subsequence, we may assume 

z > = (u',v'), in tfW) x HHm") 



z n —>z= m Ll c (M N ) x L^), 2 < q < 

Moreover z' = (u',v r ) solves the equation (11.11) . 

Now, we want to prove that z' ^ 0. If it is not true, we have z 1 = (u',v') = (0,0). We 
claim that: 

Claim: For any 1 > r > 2 — y^, 



[(O+F 1 dV mN = o(l), / IKhr 1 dV mN = o(l). 

H^n{M>r} Jm N f]{\x\>r} 

To do this, let us fix a point a E S^. Let $ G C^(A(a, Vs)) such that < $ < 1, where 
A(a, \/3) = B(a, \/3) f]M N , where B(a, \/3) is the Euclidean ball with center a and radius 

Vs. 

[ V M N U y uN y dV mN = [ [(v' n ) + }^ dV m n, for^/eH 1 ^). 
Now putting \P = $ 2 (u^) + , in the above identity, we get 

/ V m Nuy MN [^ 2 (u' n ) + ]dV m = [ [(v' n ) + ] p ^ 2 (u' n ) + dV m . 
A simple computation gives 

V H iv[$«) + ]| 2 dV MN = f [K) + ] p $ 2 «) + dV m + o(l). (5.4) 

w N Ju N 

Now using ( 15.4f) . Holder inequality and Poincare-Sobolev inequality, we get 

\\®( U n)+\\%i(M N ) 

< (LAHv'nhr 1 rfW +1 (/ H ^K.)+P +1 dv mN y+\f A{a ^ } {(v'j + r^ dv m ) p ^ 

< ^jv 2 P ll $ K)+lkHH^)ll $ K)+llHi(H^)(X 4(aiV ^ ) [(<)+F +1 dv M N)^. 

Similarly, 

< ^ 2 p ll^(<)+ll//i(H'V)||$K) + ||HMH'V)(X 4(aiV 3 ) [K) + ] 9+1 dV mN )&. 

Now, if ||$(^) + || H i( H iV) -ft and ||$(?i / „) + ||^i( H iV) -fa as n — > oo. We get 

p-i g-i 

Which is a contradiction. This implies that J mN [^(v' n ) + ] p+1 dV^N — > and J HJV [$(w'„) + ]' ?+1 cfVe-iv 
0. Since a e £ aj is arbitrary, the claim follows. 
If 1 < p, q < -j^rf, this together with the fact that 

iV + 2 



2... 



iV-2' 
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immediately gives a contradiction to (I5.3p . Which implies that z' = (u',v') ^ (0,0). 
Thus, we have 



lim I(z' n ) 

n — >oo 



> hminf(i - ^) J RN [K) + ] p+1 dV mN + (| - ^j) f m AK)+] q+1 dV mN 

> iw 

Consequently I(z') is obtained. 

To show that I°° > 0, we notice only that if z' = (u', v') ^ (0, 0) with I(z') ^ 0, then 

IV) = {\- -jrr) / [K)+] p+1 + (~ - -^t) / [KUF 1 > o. 
^ p + 1 Jm n 1 q + 1 jhf 

Proof of Theorem 11.41 By Lemma 15.2^ we know that there exists a nontrivial solution 
z = (u,v) of problem (II. ip with I(z) = Moreover, we have z = (u,v) > (0,0). 

Otherwise, we may assume that u changes sign, u = u + — it_, u + ^ 0, w_ ^ satisfying 



iVe^tt-l cIVmn = / (v+) p n_ dV^N > 0. 
w N Jw N 

So we have that w_ = 0, that is w > 0. Similarly, we have v > 0. 

□ 
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